Popular Searches
Popular Course Categories
Popular Courses

Performing Exploratory Data Analysis and Generating Analytical Reports

Performing Exploratory Data Analysis and Generating Analytical Reports

Python Data Analysis and Visualization

Performing Exploratory Data Analysis and Generating Analytical Reports in Python

Course: Python Training

Topic: Performing Exploratory Data Analysis and Generating Analytical Reports

Course Link: Python Training Course

Demo Link: Register for a Course Demo

Website: JustAcademy


1. Introduction to Exploratory Data Analysis

Exploratory Data Analysis (EDA) is the process of examining, summarizing, and visualizing data to understand its structure, discover patterns, identify relationships, and detect potential problems before conducting deeper statistical analysis or building machine learning models.

EDA helps answer important questions about a dataset: What information does it contain? Are there missing or duplicated records? Which variables have the highest values? Are there trends over time? Do certain categories perform better than others?

Python is widely used for EDA because libraries such as pandas, NumPy, Matplotlib, and Seaborn provide convenient tools for data inspection, calculations, statistical summaries, and visualization.

Objectives of EDA

  • Understand the size, structure, and data types of a dataset.
  • Identify missing values, duplicate records, and inconsistent entries.
  • Calculate descriptive statistics for numerical and categorical variables.
  • Discover patterns, trends, relationships, and unusual observations.
  • Compare performance across categories, regions, or time periods.
  • Prepare reliable information for analytical reports and business decisions.

2. Understanding Analytical Reports

An analytical report is a structured document that presents data findings, important metrics, visualizations, interpretations, and recommendations. Its purpose is to turn raw data into information that readers can understand and use.

For example, a retail sales report may explain total revenue, monthly sales trends, best-performing products, regional performance, and opportunities for improvement.

Components of an Analytical Report

Component Description
Report title Identifies the topic and scope of the analysis.
Objective Explains the question or business problem being investigated.
Dataset overview Describes the data source, time period, rows, and columns.
Data-quality assessment Documents missing values, duplicates, and data limitations.
Key performance indicators Summarizes important business measurements.
Visualizations Displays trends, distributions, comparisons, and relationships.
Findings Explains what the data reveals.
Recommendations Suggests actions supported by the findings.
Limitations Identifies assumptions, incomplete information, and constraints.

3. Python Libraries Used for EDA

3.1 pandas

Pandas is used to load, filter, clean, group, transform, and summarize structured data using Series and DataFrames.

import pandas as pd

df = pd.read_csv("sales.csv")
print(df.head())

3.2 NumPy

NumPy provides numerical arrays and mathematical functions for statistical calculations and numerical analysis.

import numpy as np

values = np.array([100, 200, 300, 400])

print("Mean:", np.mean(values))
print("Median:", np.median(values))
print("Standard deviation:", np.std(values))

3.3 Matplotlib

Matplotlib is used to create line charts, bar charts, histograms, scatter plots, and other visualizations.

import matplotlib.pyplot as plt

plt.plot([1, 2, 3, 4], [10, 15, 12, 20])
plt.title("Sample Trend")
plt.xlabel("Period")
plt.ylabel("Value")
plt.tight_layout()
plt.show()

3.4 Seaborn

Seaborn simplifies statistical visualization and supports charts such as box plots, heatmaps, histograms, and scatter plots.

import seaborn as sns
import matplotlib.pyplot as plt

sns.histplot(data=df, x="Sales", kde=True)
plt.title("Sales Distribution")
plt.tight_layout()
plt.show()

3.5 Installing Required Libraries

pip install pandas numpy matplotlib seaborn openpyxl

Run the command in your terminal or Python environment. The openpyxl package supports many Excel file operations.

4. Loading Data into Python

Data can come from CSV files, Excel workbooks, databases, APIs, and other structured sources. The loading method depends on the file format and data source.

4.1 Read a CSV File

import pandas as pd

df = pd.read_csv("sales.csv")
print(df.head())

4.2 Read an Excel File

df = pd.read_excel("sales.xlsx")
print(df.head())

4.3 Read a Specific Excel Sheet

df = pd.read_excel(
    "sales.xlsx",
    sheet_name="Monthly Sales"
)

4.4 Load Data from SQLite

import sqlite3
import pandas as pd

connection = sqlite3.connect("business.db")

try:
    query = "SELECT * FROM sales"
    df = pd.read_sql_query(query, connection)
finally:
    connection.close()

print(df.head())

When loading data, verify the file path, encoding, column names, and date formats. For large datasets, consider loading only the required columns or processing the data in chunks.

5. Inspecting Dataset Structure

Before calculating metrics, inspect the dataset to understand its dimensions, columns, data types, and sample records.

5.1 Display the First and Last Rows

print(df.head())
print(df.tail())

head() displays the first five rows by default, while tail() displays the last five rows.

5.2 Check the Number of Rows and Columns

print("Dataset shape:", df.shape)
print("Rows:", df.shape[0])
print("Columns:", df.shape[1])

5.3 Inspect Column Names

print(df.columns.tolist())

Column names should be clear and consistent. Remove unwanted whitespace when necessary:

df.columns = df.columns.str.strip()

5.4 Inspect Data Types and Non-Missing Values

df.info()

This method displays column names, data types, non-missing counts, and memory usage. It helps identify columns that may have been imported using incorrect data types.

5.5 Check Unique Values

for column in df.columns:
    print(f"\nColumn: {column}")
    print("Unique values:", df[column].nunique(dropna=True))

Unique-value counts help identify categorical variables, potential identifiers, and columns with very few distinct values.

6. Descriptive Statistics

Descriptive statistics summarize the central tendency, spread, and range of values in a dataset.

6.1 Generate Summary Statistics

print(df.describe())

For numeric columns, describe() typically returns the count, mean, standard deviation, minimum, quartiles, and maximum.

6.2 Include Categorical Columns

print(df.describe(include="all"))

For categorical columns, the summary may include the number of unique values, the most frequent value, and its frequency.

6.3 Calculate Individual Statistics

print("Mean:", df["Sales"].mean())
print("Median:", df["Sales"].median())
print("Minimum:", df["Sales"].min())
print("Maximum:", df["Sales"].max())
print("Standard deviation:", df["Sales"].std())

6.4 Understanding Common Statistics

Statistic Meaning Application
Mean Arithmetic average. Average sales per transaction.
Median Middle value in ordered data. Typical order value when outliers exist.
Mode Most frequently occurring value. Most popular product category.
Minimum Smallest observed value. Lowest transaction amount.
Maximum Largest observed value. Highest transaction amount.
Standard deviation Measures variability around the mean. Variation in sales values.
Quartiles Divide ordered observations into four sections. Understanding data distribution.

7. Assessing Data Quality

Data-quality checks are important because missing values, duplicate records, and incorrect entries can distort statistics and visualizations.

7.1 Count Missing Values

missing_counts = df.isna().sum()
print(missing_counts)

7.2 Calculate Missing-Value Percentages

missing_percentages = (
    df.isna().mean() * 100
).round(2)

print(missing_percentages)

7.3 Detect Duplicate Rows

print("Duplicate rows:", df.duplicated().sum())

7.4 Review Categorical Values

print(df["Category"].value_counts(dropna=False))
print(df["Category"].unique())

7.5 Check Data Types

print(df.dtypes)

Use the results to determine whether values need to be standardized, converted, filled, or investigated. Do not automatically remove every unusual record because some unusual values may be legitimate.

8. Univariate Analysis

Univariate analysis examines one variable at a time. It helps identify distributions, common values, variation, and potential outliers.

8.1 Analyze a Numeric Variable

print(df["Sales"].describe())
print("Median sales:", df["Sales"].median())
print("Unique sales values:", df["Sales"].nunique())

8.2 Create a Histogram

import matplotlib.pyplot as plt

df["Sales"].dropna().plot(
    kind="hist",
    bins=20,
    edgecolor="black"
)

plt.title("Distribution of Sales")
plt.xlabel("Sales Amount")
plt.ylabel("Frequency")
plt.tight_layout()
plt.show()

A histogram groups numeric values into intervals. It helps identify skewness, clusters, multiple peaks, and potential outliers.

8.3 Analyze a Categorical Variable

category_counts = df["Category"].value_counts()
print(category_counts)

8.4 Create a Bar Chart

import matplotlib.pyplot as plt

category_counts = df["Category"].value_counts()

category_counts.plot(kind="bar")
plt.title("Records by Category")
plt.xlabel("Category")
plt.ylabel("Number of Records")
plt.xticks(rotation=45)
plt.tight_layout()
plt.show()

Bar charts help compare counts or summary values across categories.

9. Bivariate Analysis

Bivariate analysis examines two variables to understand their relationship or compare their behavior.

9.1 Calculate Correlation

correlation = df[
    ["AdvertisingSpend", "Sales"]
].corr()

print(correlation)

Correlation measures the strength and direction of a linear association between two numeric variables. A coefficient near 1 indicates a strong positive linear relationship, while a coefficient near -1 indicates a strong negative linear relationship. A value near zero indicates little linear association.

Correlation does not prove causation. Other variables, unusual observations, or nonlinear relationships may influence the result.

9.2 Create a Scatter Plot

import seaborn as sns
import matplotlib.pyplot as plt

sns.scatterplot(
    data=df,
    x="AdvertisingSpend",
    y="Sales"
)

plt.title("Advertising Spend vs Sales")
plt.tight_layout()
plt.show()

A scatter plot helps reveal clusters, nonlinear patterns, and unusual observations.

9.3 Compare a Numeric Variable Across Categories

category_sales = df.groupby("Category")["Sales"].mean()
print(category_sales)

This calculates average sales for each category. If the data is skewed or category sizes differ substantially, also examine medians, sample sizes, and distributions.

10. Multivariate Analysis

Multivariate analysis examines several variables together. It can reveal patterns that are not visible when analyzing individual columns separately.

10.1 Create a Correlation Matrix

numeric_df = df.select_dtypes(include="number")
correlation_matrix = numeric_df.corr()

print(correlation_matrix)

10.2 Visualize Correlations with a Heatmap

import seaborn as sns
import matplotlib.pyplot as plt

sns.heatmap(
    correlation_matrix,
    annot=True,
    cmap="coolwarm",
    fmt=".2f"
)

plt.title("Correlation Matrix")
plt.tight_layout()
plt.show()

A heatmap makes it easier to compare pairwise correlations among numeric variables. Interpret these results alongside sample sizes, data quality, and domain knowledge.

10.3 Compare Multiple Groups

summary = df.groupby(
    ["Region", "Category"]
).agg(
    Total_Sales=("Sales", "sum"),
    Average_Sales=("Sales", "mean"),
    Transaction_Count=("Sales", "count")
)

print(summary)

This example compares categories within regions and calculates several metrics in one operation.

11. Time-Series Exploratory Data Analysis

Time-series analysis examines observations recorded over time, such as daily revenue, monthly sales, or hourly website traffic. It helps identify trends, seasonal patterns, fluctuations, and unusual events.

11.1 Convert Dates to Datetime

df["OrderDate"] = pd.to_datetime(
    df["OrderDate"],
    errors="coerce"
)

df = df.sort_values("OrderDate")

11.2 Group Sales by Month

monthly_sales = (
    df.set_index("OrderDate")["Sales"]
    .resample("ME")
    .sum()
)

print(monthly_sales)

The ME frequency groups values by calendar month-end in current pandas versions. Confirm that dates and sales values are valid before interpreting totals.

11.3 Visualize Monthly Sales

import matplotlib.pyplot as plt

monthly_sales.plot(kind="line", marker="o")

plt.title("Monthly Sales Trend")
plt.xlabel("Month")
plt.ylabel("Total Sales")
plt.tight_layout()
plt.show()

A line chart helps identify upward or downward trends. Seasonal patterns should be investigated using sufficient historical data rather than inferred from only a few observations.

12. Grouping Data and Calculating Business Metrics

The pandas groupby() method divides records into groups and calculates statistics for each group. It is useful for comparing regions, products, departments, customer segments, and sales channels.

12.1 Calculate Total Sales by Region

regional_sales = df.groupby("Region")["Sales"].sum()
print(regional_sales)

12.2 Calculate Multiple Metrics

summary = df.groupby("Region").agg(
    Total_Sales=("Sales", "sum"),
    Average_Sales=("Sales", "mean"),
    Minimum_Sales=("Sales", "min"),
    Maximum_Sales=("Sales", "max"),
    Transaction_Count=("Sales", "count")
)

print(summary)

The transaction count in this example counts non-missing sales values. It may differ from the number of distinct orders if one order contains multiple product lines.

12.3 Identify Best-Selling Products

product_sales = (
    df.groupby("Product")["Sales"]
    .sum()
    .sort_values(ascending=False)
)

print(product_sales.head(10))

This identifies the ten products with the highest total sales. Revenue alone does not measure profitability, so include cost or profit metrics when evaluating overall performance.

13. Calculating Key Performance Indicators

Key Performance Indicators (KPIs) are measurable values used to evaluate progress toward a business objective. The correct KPI depends on the business model and the question being investigated.

KPI Meaning Calculation
Total Revenue Total revenue generated. Sum of revenue values.
Average Order Value Average revenue per order. Total revenue divided by valid order count.
Order Count Number of distinct orders. Count of unique order IDs.
Profit Revenue minus applicable costs. Total revenue minus total cost.
Profit Margin Profit as a percentage of revenue. Profit / Revenue × 100.
Growth Rate Relative change between two periods. (Current - Previous) / Previous × 100.

13.1 Calculate Total Revenue

total_revenue = df["Sales"].sum()
print(f"Total Revenue: {total_revenue:,.2f}")

13.2 Calculate Average Order Value

# Valid when each row represents exactly one order
average_order_value = df["Sales"].mean()

print(f"Average Order Value: {average_order_value:,.2f}")

If one order contains multiple product rows, first aggregate sales by order ID and then calculate the average of order totals. Otherwise, the calculation may represent average line-item sales rather than average order value.

13.3 Calculate Profit Margin

total_revenue = df["Revenue"].sum()
total_profit = df["Profit"].sum()

if total_revenue != 0:
    profit_margin = total_profit / total_revenue * 100
    print(f"Profit Margin: {profit_margin:.2f}%")
else:
    print("Profit margin cannot be calculated because revenue is zero.")

13.4 Calculate Growth Rate

previous_period = 100000
current_period = 125000

if previous_period != 0:
    growth_rate = (
        (current_period - previous_period)
        / previous_period
    ) * 100

    print(f"Growth Rate: {growth_rate:.2f}%")

The example shows 25% growth from 100,000 to 125,000. A zero or negative baseline requires careful interpretation according to the metric.

14. Detecting Potential Outliers

An outlier is an observation that differs substantially from most other observations. It may indicate a data-entry error, an unusual but valid event, or an important business occurrence.

14.1 Create a Box Plot

import seaborn as sns
import matplotlib.pyplot as plt

sns.boxplot(data=df, x="Sales")
plt.title("Sales Outlier Inspection")
plt.tight_layout()
plt.show()

14.2 Use the Interquartile Range Method

The interquartile range (IQR) is the difference between the third quartile (Q3) and first quartile (Q1). A common exploratory rule flags values below Q1 − 1.5 × IQR or above Q3 + 1.5 × IQR.

q1 = df["Sales"].quantile(0.25)
q3 = df["Sales"].quantile(0.75)

iqr = q3 - q1

lower_bound = q1 - 1.5 * iqr
upper_bound = q3 + 1.5 * iqr

outliers = df[
    (df["Sales"] < lower_bound)
    | (df["Sales"] > upper_bound)
]

print(outliers)

This method flags potential outliers; it does not prove that the records are incorrect. Investigate them before deciding whether to retain, transform, or exclude them.

15. Creating Effective Visualizations

Visualizations communicate analytical findings clearly. Each chart should answer a specific question and include a descriptive title, readable labels, appropriate units, and a suitable scale.

Chart Type Purpose Example
Bar chart Compare categories. Revenue by region.
Line chart Show changes over time. Monthly sales trends.
Histogram Explore numeric distributions. Order value distribution.
Box plot Compare distributions and identify potential outliers. Sales across categories.
Scatter plot Explore relationships between numeric variables. Advertising spending versus sales.
Heatmap Display a matrix of values. Correlation matrix.
Stacked bar chart Compare totals and category composition. Sales channels by region.

15.1 Create a Horizontal Bar Chart

import matplotlib.pyplot as plt

region_sales = df.groupby("Region")["Sales"].sum()

region_sales.sort_values().plot(kind="barh")

plt.title("Total Sales by Region")
plt.xlabel("Total Sales")
plt.ylabel("Region")
plt.tight_layout()
plt.show()

A good chart avoids unnecessary decoration and makes comparisons easy. If many categories exist, consider sorting the bars or displaying only the most relevant categories.

16. Building an Analytical Summary Table

A summary table brings important metrics together in one place. It can be displayed in a notebook or exported to CSV, Excel, or HTML.

summary_table = pd.DataFrame({
    "Metric": [
        "Total Sales",
        "Average Sales",
        "Median Sales",
        "Maximum Sales",
        "Minimum Sales",
        "Missing Sales Values"
    ],
    "Value": [
        df["Sales"].sum(),
        df["Sales"].mean(),
        df["Sales"].median(),
        df["Sales"].max(),
        df["Sales"].min(),
        df["Sales"].isna().sum()
    ]
})

print(summary_table)

This example assumes that the dataset contains a numeric column named Sales. Adapt the metrics and column names to the dataset being analyzed.

17. Exporting Analytical Reports

Python can export analytical results into several formats. CSV is useful for tabular data, Excel supports multiple worksheets, HTML is suitable for browser-based reports, and image files can store charts for presentations.

17.1 Export a Summary to CSV

summary_table.to_csv(
    "analytical_summary.csv",
    index=False
)

17.2 Export Results to Excel

with pd.ExcelWriter("analytical_report.xlsx") as writer:
    summary_table.to_excel(
        writer,
        sheet_name="Summary",
        index=False
    )

    regional_sales.rename("Total Sales").to_frame().to_excel(
        writer,
        sheet_name="Regional Sales"
    )

The workbook contains separate worksheets for the overall summary and regional sales. Install a compatible Excel-writing engine, such as openpyxl, if needed.

17.3 Export a Table as HTML

html_table = summary_table.to_html(
    index=False,
    border=1
)

with open(
    "analytical_summary.html",
    "w",
    encoding="utf-8"
) as file:
    file.write("<h1>Analytical Summary</h1>\\n")
    file.write(html_table)

The generated file can be opened in a web browser. A complete report should also include the objective, data-quality results, charts, interpretations, and recommendations.

17.4 Save a Chart as an Image

import matplotlib.pyplot as plt

region_sales.plot(kind="bar")

plt.title("Total Sales by Region")
plt.xlabel("Region")
plt.ylabel("Sales")
plt.tight_layout()

plt.savefig(
    "regional_sales.png",
    dpi=150,
    bbox_inches="tight"
)

plt.close()

Save the figure before closing it. Select an appropriate resolution for the report or presentation.

18. Creating a Reusable EDA Report

A reusable function can automate basic data-quality checks and descriptive statistics for different datasets.

import pandas as pd

def generate_eda_report(df, output_prefix="eda_report"):
    if df.empty:
        raise ValueError("The dataset contains no rows.")

    overview = pd.DataFrame({
        "Metric": [
            "Number of Rows",
            "Number of Columns",
            "Duplicate Rows"
        ],
        "Value": [
            len(df),
            len(df.columns),
            int(df.duplicated().sum())
        ]
    })

    quality = pd.DataFrame({
        "Data Type": df.dtypes.astype(str),
        "Missing Count": df.isna().sum(),
        "Missing Percentage": (
            df.isna().mean() * 100
        ).round(2),
        "Unique Values": df.nunique(dropna=True)
    })

    statistics = df.describe(
        include="all"
    ).transpose()

    overview.to_csv(
        f"{output_prefix}_overview.csv",
        index=False
    )

    quality.to_csv(
        f"{output_prefix}_quality.csv"
    )

    statistics.to_csv(
        f"{output_prefix}_statistics.csv"
    )

    html = f"""
    <html>
    <head>
        <meta charset="utf-8">
        <title>Exploratory Data Analysis Report</title>
    </head>
    <body>
        <h1>Exploratory Data Analysis Report</h1>
        <h2>Dataset Overview</h2>
        {overview.to_html(index=False, border=1)}
        <h2>Data Quality</h2>
        {quality.to_html(border=1)}
        <h2>Descriptive Statistics</h2>
        {statistics.to_html(border=1)}
    </body>
    </html>
    """

    with open(
        f"{output_prefix}.html",
        "w",
        encoding="utf-8"
    ) as file:
        file.write(html)

    return overview, quality, statistics


# Example usage
df = pd.read_csv("sales.csv")

overview, quality, statistics = generate_eda_report(df)

print(overview)
print(quality)
print(statistics)

This function generates an HTML report and three CSV files. For production use, expand it to document the data source, analysis period, cleaning decisions, metric definitions, charts, and business interpretation. If column values are untrusted and the report will be published, escape user-provided text before inserting it into HTML.

19. Interpreting Findings and Writing Recommendations

An analytical report should explain what the results mean rather than merely listing numbers. It should distinguish observations from possible explanations and recommendations.

Example of an Analytical Finding

Observation: The regional summary indicates that Region A has the highest total sales among the regions included in the dataset.

Interpretation: Region A contributes the largest share of recorded revenue during the selected period. This may reflect a larger customer base, higher transaction values, or a different product mix.

Recommendation: Compare order volume, average order value, product mix, and profit margin across regions before deciding whether to increase marketing investment in Region A.

Limitation: The report covers only the available records and time period. It does not establish why one region performs better than another.

Questions an Analytical Report Should Answer

  • What question does the analysis address?
  • Which dataset and time period were used?
  • What are the most important metrics?
  • Which trends or differences are visible?
  • Are findings affected by missing data or unusual observations?
  • What limitations could influence the conclusions?
  • What practical action is supported by the evidence?

20. Best Practices for EDA and Analytical Reporting

  • Define the business question before beginning analysis.
  • Preserve the original dataset before cleaning.
  • Verify data types, units, time zones, and date ranges.
  • Check missing values and duplicates before interpreting results.
  • Use descriptive statistics and visualizations together.
  • Choose chart types that match the analytical question.
  • Do not interpret correlation as proof of causation.
  • Define metrics and denominators clearly.
  • Document exclusions, assumptions, and transformations.
  • Use consistent number formats and readable chart labels.
  • Protect confidential and personally identifiable information.
  • Make the analysis reproducible with documented Python code.
  • Verify exported files and report calculations before sharing.

21. Common Errors and Troubleshooting

Problem Possible Cause Solution
FileNotFoundError Incorrect file path or unavailable file. Check the working directory and file location.
KeyError for a column The requested column does not exist. Inspect df.columns and verify spelling.
TypeError during calculations A numeric column contains text values. Inspect values and convert the column appropriately.
Unexpected totals Duplicates or incorrect aggregation rules. Check record uniqueness and the meaning of each row.
Charts omit observations Missing values or incompatible data types. Inspect missing values and confirm plotting behavior.
Correlation contains missing values Insufficient valid observations or constant columns. Check sample sizes, missing values, and variable variation.
Excel export fails An Excel-writing engine is unavailable. Install a compatible package, such as openpyxl.
HTML report is difficult to read The table is too wide or contains too many columns. Select relevant columns or divide the results into sections.

22. Practical Exercises

Exercise 1: Dataset Inspection

Load a sales CSV file and display its first ten rows, dimensions, column names, data types, and descriptive statistics.

Exercise 2: Data Quality Report

Calculate missing-value counts and percentages, count duplicate rows, and list unique values in categorical columns.

Exercise 3: Univariate Analysis

Generate a histogram for sales amounts and a bar chart for product categories. Write two observations based on the results.

Exercise 4: Bivariate Analysis

Create a scatter plot of advertising spending and sales. Calculate their correlation and explain why the result does not prove causation.

Exercise 5: Regional Performance

Calculate total sales, average sales, and transaction counts by region. Identify the highest-performing region for the selected metric.

Exercise 6: Analytical Reporting

Export a dataset overview, data-quality summary, descriptive statistics, and at least one chart. Write a conclusion and two evidence-based recommendations.

23. Mini Project: Retail Sales EDA and Reporting

Project Objective: Analyze a retail sales dataset and produce a report covering business performance, data quality, product performance, and sales trends.

Suggested Dataset Columns

  • OrderID — unique order identifier.
  • OrderDate — transaction date.
  • Product — product name or identifier.
  • Category — product category.
  • Region — sales region.
  • Quantity — units sold.
  • Sales — recorded sales amount.
  • Profit — recorded profit, if available.

Project Steps

  1. Load the dataset and preserve an original copy.
  2. Inspect dimensions, column names, and data types.
  3. Check missing values, duplicates, and invalid dates.
  4. Validate quantity, sales, and profit values against business rules.
  5. Calculate descriptive statistics for numeric columns.
  6. Visualize sales distributions and monthly trends.
  7. Compare sales and profit by region and category.
  8. Calculate relevant KPIs using clearly defined formulas.
  9. Export summary tables and visualizations.
  10. Create an HTML or Excel report with findings, limitations, and recommendations.

Example: Regional and Category Summary

import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv("retail_sales.csv")

df["OrderDate"] = pd.to_datetime(
    df["OrderDate"],
    errors="coerce"
)

regional_summary = df.groupby("Region").agg(
    Total_Sales=("Sales", "sum"),
    Average_Sales=("Sales", "mean"),
    Order_Line_Count=("Sales", "count")
).sort_values("Total_Sales", ascending=False)

category_summary = df.groupby("Category").agg(
    Total_Sales=("Sales", "sum"),
    Total_Quantity=("Quantity", "sum")
).sort_values("Total_Sales", ascending=False)

print(regional_summary)
print(category_summary)

regional_summary.to_csv("regional_summary.csv")
category_summary.to_csv("category_summary.csv")

regional_summary["Total_Sales"].sort_values().plot(
    kind="barh"
)

plt.title("Total Sales by Region")
plt.xlabel("Total Sales")
plt.ylabel("Region")
plt.tight_layout()
plt.savefig("sales_by_region.png", dpi=150)
plt.close()

The example assumes the specified columns exist in the CSV file. The Order_Line_Count counts non-missing sales values, not necessarily distinct orders. If one order contains multiple product lines, use the distinct OrderID count for order-level reporting.

24. Interview Questions and Answers

Q1. What is Exploratory Data Analysis?

Answer: EDA is the process of examining data using descriptive statistics, visualizations, and quality checks to understand its structure, distributions, patterns, relationships, and anomalies.

Q2. Which Python libraries are commonly used for EDA?

Answer: Pandas and NumPy support data manipulation and numerical calculations, while Matplotlib and Seaborn are commonly used for visualization.

Q3. What does pandas describe() do?

Answer: It generates descriptive statistics. Numeric columns typically include count, mean, standard deviation, minimum, quartiles, and maximum.

Q4. What is the difference between univariate and bivariate analysis?

Answer: Univariate analysis studies one variable at a time, while bivariate analysis examines two variables to explore their relationship or compare their behavior.

Q5. What is a correlation matrix?

Answer: It is a table of pairwise correlation coefficients for selected variables. It helps identify linear associations but does not establish causation.

Q6. When should a histogram be used?

Answer: A histogram examines the distribution of a numeric variable, including its spread, skewness, clusters, and potential outliers.

Q7. What is the purpose of groupby()?

Answer: It groups records according to one or more keys and supports calculations such as sums, means, counts, and medians for each group.

Q8. What is a KPI?

Answer: A Key Performance Indicator is a measurable value used to evaluate progress toward a defined business objective.

Q9. How can Python generate an analytical report?

Answer: Use pandas to calculate metrics and tables, Matplotlib or Seaborn to create charts, and export functions such as to_csv(), to_excel(), and to_html() to generate report files.

Q10. Why is data cleaning important before EDA?

Answer: Missing values, duplicates, incorrect types, and invalid entries can distort statistics and visualizations. Cleaning and documenting these issues improves the reliability of findings.

Q11. What is the difference between a finding and a recommendation?

Answer: A finding describes what the data shows. A recommendation suggests an action based on the finding, business context, and relevant limitations.

Q12. How can an EDA report be made reproducible?

Answer: Preserve the source data, document the steps, define metrics consistently, record assumptions, and use scripts or notebooks that can be rerun on the intended dataset.

25. Quick Revision Checklist

  • Define the business question before starting analysis.
  • Load data and inspect dimensions, columns, and data types.
  • Review descriptive statistics and unique-value counts.
  • Check missing values, duplicates, and invalid records.
  • Use univariate analysis to examine individual variables.
  • Use bivariate and multivariate analysis to explore relationships.
  • Create appropriate histograms, bar charts, line charts, scatter plots, and heatmaps.
  • Use grouping and aggregation to compare categories.
  • Calculate KPIs with clear definitions and correct denominators.
  • Investigate potential outliers instead of automatically deleting them.
  • Export summary tables and charts in suitable formats.
  • Document findings, assumptions, limitations, and recommendations.
  • Verify the report before sharing it.

26. Conclusion

Exploratory Data Analysis is a fundamental step in the Python data-analysis workflow. It combines data inspection, descriptive statistics, data-quality checks, grouped calculations, and visualizations to help analysts understand what a dataset contains and which questions deserve further investigation.

Generating an analytical report extends this process by organizing results into meaningful metrics, charts, findings, and recommendations. A useful report explains the context, defines measurements, acknowledges limitations, and supports decisions with evidence rather than simply displaying numbers.

Additional Learning Resources

whatsapp